Flipping the Classroom: 

An Empirical Study Examining Student Learning 


Roland J. Sparks, DBA 

Associate Professor Business 
College of Business 
Johnson & Wales University 
Charlotte, North Carolina 

ABSTRACT 

Flipping the classroom is the latest reported teaching technique to improve student learning at all levels. Prior studies 
showed significant increases in learning by employing this technique. However, an examination ofthe previous studies 
indicates significantflaws in the testing procedure controls. Moreover, most studies were based on anecdotal observa¬ 
tions and not quantitative methods of analysis. 

This studied examined the true learning improvements attributed to the flipped classroom method. The results indi¬ 
cate thatflipping the classroom did improve the test scores for 14% of the students and 88% of the students self-reported 
that the flipped classroom resulted in more effective learning. However, overall class test scores did not support that 
flipping the classroom improved the entire class. Infact, 81.5% of the students showed no significant improvement and 
3.7% showed lower test scores using theflipped classroom method. 

The flipped classroom does have the benefit of increasing time on task for the student by using technology to increase 
learning time outside the classroom. However, this technique is only effective because students spend more time learn¬ 
ing the material. Any method that gives more time to learn the material will result in increased learning. The true 
value of theflipped classroom appears to be not in the method but in the use of technology to increase the time students 
spend learning. 


INTRODUCTION 

“Flipping the classroom” are the latest buzzwords in aca¬ 
demics from K-12 to college teaching. Flipping the class¬ 
room is where instruction is moved outside the classroom 
giving time in the class to increase the depth of learning. 
The current technique is to place resources online, such as 
video lectures, allowing the student to learn the material 
before entering the classroom. The classroom time is now 
spent working with the students and answering questions 
to further student knowledge. 

There are repeated claims that this technique improved 
student learning at all levels. However, an examination 
of peer reviewed research articles shows no empirical sup¬ 
port for these claims. Moreover, the results claimed in 
non-peer reviewed articles are suspicious because the re¬ 
search data is undocumented and only descriptive statis¬ 
tics were reported. 

This paper examines the effectiveness of flipping the 
classroom on a freshman college class under controlled 
conditions using a paired sampling technique. Through 
controlled standardized testing, the learning results of 
flipping the classroom are properly evaluated. 


LITERATURE REVIEW 

Although flipping the classroom may have taken place in 
similar forms, the first published article on flipping the 
classroom using modern techniques of online videos was 
in 2002 (Foertsch, Moses, Strikwerda, & Litzkow, 2002). 
In the study, online videos and accompanying material 
replaced traditional lectures. Lecture time was used for 
interactive discussion and problem solving. Students self 
reported liking the learning method better than the large 
traditional lecture classroom setting. However, no re¬ 
search was done on whether this improved student learn¬ 
ing. 

The Khan Academy in 2006 progressed the idea of 
moving learning outside the classroom by founding a 
non-profit educational website that offered 4,000 micro 
lectures on various academic subjects (Khan Academy, 
2013). The venture was funded by the Gates Foundation 
to make education more available to the public through 
online resources. Although the Khan Academy does not 
offer the flipped classroom teaching, the Khan videos 
were the basis of many flipped classroom experiments that 
later developed. 
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With the flipped learning method developing, Strayer 
(2007), examined how students felt about the new learn¬ 
ing environment. In his dissertation, Strayer surveyed 
college level students in statistics courses at Ohio State 
University. Although Strayer found students were un¬ 
comfortable with the learn methods, Strayer never exam¬ 
ined whether student learning increased using the flipped 
classroom method. 

In 2012, the research on flipping the classroom took oft, 
mostly due to the work of Bergmann and Sams in their 
journal article “Before you flip consider this” and their 
book, Flip Your Classroom. The journal article detailed 
the history of flipping the classroom and gave guide¬ 
lines for academics on how they could use this technique 
(Bergmann & Sams, Before you flip, consider this, 2012). 
A full-length book followed up the journal article giving 
examples of the flip technique (Bergmann & Sams, Flip 
Your Classroom, 2012). The articles and book gave de¬ 
tailed instructions to implement a flipped classroom and 
why flipping would improve student learning. The specific 
reasons given for flipping the classroom are: students don’t 
feel classroom time is being dumbed down, the digital 
world changed how students learn, students are becoming 
multitaskers, increased student-teacher interaction, in¬ 
creased student-to-student interaction, increased student- 
to-world interaction, students can work ahead or at own 
pace, instruction time is flexible to fit into student’s sched¬ 
ule and when student is most alert, gives instructor more 
one-on-one time with struggling students, aids special 
needs students by allowing repetition of lecture or slow¬ 
ing of lecture, allows instructors to cover more material in 
class, leverages technology to increase student interaction 
or active learning, allows teachers to get to know students 
better, allows students of different abilities to learn in the 
same class, aids in classroom management, allows parents 
to help and monitor student activity, and aids students 
who miss classes due to illness or other factors (Bergmann 
& Sams, Flip Your Classroom, 2012). 

Brunsell (2011), verified some of Bergmann and Sams 
work finding that flipping his chemistry class increased 
one-on-one interaction and student learning. However, 
Brunsell’s work relied on anecdotal observation and not 
quantitative methods of analysis. In 2012, Berrett de¬ 
scribed the process of flipping the classroom and reported 
that it increases interactive learning, group work and peer 
instruction. However, he provided no research to confirm 
his statements. 

From 2010-2012, there were a number of articles that 
reported on personal experiences with flipping the class¬ 
room. Alvarez (2010), described its use in high school 
physics courses. Ogurek (2010), used an early Dunn and 
Dunn learning style model in elementary school classes. 


Corbyn (2012), used MOOC (Massive Open Online 
Courses) as the out of class online lectures. Stephens 
(2012), used social media to supplement out of class learn¬ 
ing. In addition, Semple (2013), examined teaching li¬ 
brary science in a flipped version. However, none of these 
works analyzed if student learning increased because of 
using this new method. 

It seemed that flipping the classroom was generating its 
own buzz in learning journals. Both the Economist (Flip¬ 
ping the Classroom, 2011) and Phi Delta Kappan (Flip¬ 
ping Classrooms, 2011) magazines did short stories about 
the techniques. The magazines touted this as the new and 
upcoming method of learning but provided little support¬ 
ing evidence of its effectiveness. 

The teaching website www.techlearning.com attempted 
to quantify the impact that flipping the classroom had 
on teachers and students through an online survey. Their 
self reported results indicated that teacher job satisfaction 
improved by 88%, student satisfaction improved by 80% 
and 67% of students reported better test scores (What Do 
Teachers Who’ve Flipped Their Classrooms Have to Re¬ 
port, 2012). However, no data or testing procedures were 
provided in the one page graphic summary of the results 
they published. 

All this hype appears to be based on one research study 
that showed student learning increased in a flipped class¬ 
room. The study is Fulton’s (2012) journal article on “The 
Flipped Classroom: Transforming Education at Byron 
High School.” A review of this study showed a serious 
fault in controlling related variables. In 2010, Byron High 
School was trying to increase student scores on standard¬ 
ized exams without increasing cost to the school. The 
teachers flipped the classroom and developed online in¬ 
structional videos for students to watch before class. Then 
during class, teachers did more one-on-one problem solv¬ 
ing. The results reported a 9.8% increase in calculus scores, 
a 6.1% increase in precalculus scores and a 5.1% increase 
in algebra scores. In addition, end of grade State math 
competency scores increased from 29.9% to 65.6% using 
the flipped classroom (Fulton, The Flipped Classroom: 
Transforming Education at Byron High School, 2012). 
On the surface, these results seem remarkable. However, 
there was also a change in textbooks at the time. Before 
the flipped classroom experiment, teachers complained 
the math textbooks being used were outdated and did not 
contain the material that the State exam tested. When the 
teachers created the instructional videos, they based the 
instruction on the material that was on the State exam 
and not what was in the outdated textbook. Therefore, 
one group used outdated textbooks that did not cover the 
material being tested. The other group used the online 
instruction tailored to the exam questions. This major 


66 


Fall 2013 (Volume 9 Issue 2) 



Flipping the Classroom: An Empirical Study Examining Student Learning 


change in instructional quality is not accounted for in the 
research. One is left wondering if the improvement was 
due to better instructional material or the teaching tech¬ 
nique of flipping the classroom. 

The Fulton study is also suspect because it did not use 
any analysis technique in reporting the results. The re¬ 
sults were reported as simple descriptive statistics of class 
means. However, an examination of the available data in¬ 
dicates wide variances within the control and test groups. 
The mean scores varied over time and the control and test 
group results crossed at several points. This wide swing in 
variance would probably cause a t-test of means to be in¬ 
significant. 

The literature review strongly indicates that no real re¬ 
search has been done to verify that flipping the classroom 
increases student learning. Most articles only describe the 
experience or are reprints. This has lead to a split opin¬ 
ion in academia if this technique is beneficial to student 
learning. Some critics attacked the technique on several 
fronts to include the digital divide issue that students do 
not have necessary access at home and that increasing 
the school workload at homework negatively affect the 
student and family time (Nielsen, 2012). Based on the 
conflict and lack of literature support, it is important to 
empirically examine if flipping the classroom increases 
student learning. 

METHODOLOGY 

Increased learning from flipping the classroom takes place 
in two main areas, the pre-class work where students view 
online videos and material and the advanced in-class work 


that furthers the understanding of the material. The pre¬ 
class learning is similar to any online program available. 
Therefore, the added learning, espoused in the flipped 
method, occurs during the advanced in-class time. 

To accurately measure the increase in learning taking 
place in class, a pre and post-test technique is used. Start¬ 
ing three days before the class, students can access the 
Blackboard online learning system and view videos and 
online material at their own pace. In addition, the text¬ 
book material is always available for the student. Within 
that three-day period, the students take a standardized 
exam on the material at a time of their choosing. The 
students then attend the in-class learning session. In the 
two-hour class, a short (10-minute) overview of the key 
material is given. Students then work in groups to address 
the advanced application of the material. This is done 
through group work, simulations, case studies and prob¬ 
lem sets. The instructor works with groups and individu¬ 
als one-on-one to answer questions and direct learning to 
key objectives. Selected groups then present their work to 
the class for open class discussion defending their optimal 
solutions. Class size was kept small (n=30) to allow the in¬ 
structor enough interaction time. Within three days after 
the in-class time, the student takes a standardized post¬ 
test on the material. Figure 1 represents the study process. 

To neutralize grading bias, the pre and post-test are mul¬ 
tiple-choice questions taken randomly from the book 
publishers test bank. Analysis of the results is done using 
multiple sources of measure. For quantitative results and 
to analyze overall success, the class mean of the pre and 
post-tests are analyzed using a paired two-sample means 
t-test. Further breakdown of the results examine each in- 


Figure 1 

Flipping the Classroom Testing Method 
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dividual student’s pre and post-tests using the same-paired 
two-sample means t-test. This yields the percentage of 
students who showed increased learning from the flipped 
classroom technique. Qualitative results are obtained 
using two sources. Instructor observations are the first 
source. During the in-class time, the instructor observes 
learning demonstrated by the students. Secondly, students 
were given an end of class survey asking to rate the class, 
learning style and knowledge acquired. The survey con¬ 
tains both numerically scored and open ended questions 
to gain further insights. 

RESULTS AND ANALYSIS 

The total number of students who completed the study 
was 27. The study contained 21 pre and 21 post-tests giv¬ 
ing 567 control data points and 567 testing data points for 
the paired sample testing. Each lesson module correlated 
with one chapter of the course textbook, which averaged 
approximately 20 pages of material in the freshman level 
marketing field. The course length was 11 weeks with two, 
two-hour class meetings per week for a total of 44 hours 
of in-class flipped instruction. Table 1 lists the results for 
the individual students and Table 2 is a summary of class 
results. 

The results indicate that 14.8% of the students showed 
significant knowledge gains from the flipped classroom 
instruction and 81.5% showed no significant improve¬ 
ment. There was even one student that showed a signifi¬ 
cant decrease in performance from the flipped classroom. 

The overall class mean increased by 1.48 percentage points 
but the increase was not significant at the 0.05 level. 

Observation by the instructor indicated the following: 
students were better prepared to discuss material in class, 
students appeared to show deeper understanding of the 
material based on their verbal discussions of the topic, and 
groups presented and defended case solutions better than 
previous classes not using a flipped classroom. 

Open ended questions on the student survey indicated 
that 88% of the students felt the flipped classroom was a 
more effective learning style. Students cited an increased 
interest in learning, more in-class interaction and class 
projects that resulted in a deeper understanding of the 
material. 

The student survey contained eight critical questions to 
access how a flipped classroom affected the student. Scor¬ 
ing for the Likert scale responses were compared to pre¬ 
vious sores for this class and the instructor. Table 3 ex¬ 
amines the eight questions and if the flipped classroom 
scored better or worse than previous non-flipped class¬ 
room courses. 


Table 1 

Summary of Individual Results 

Student 

Pre-Test 

Mean 

Post-Test 

Mean 

Mean 

Improvement 

t-test 

Significance 

1 

76.75 

81.00 

4.25 

0.158 

2 

74.00 

80.53 

6.53* 

0.037 

3 

69.25 

66.50 

-2.75 

0.226 

4 

50.42 

50.53 

0.11 

0.494 

5 

69.33 

67.78 

-1.56 

0.362 

6 

82.75 

87.38 

4.63 

0.229 

7 

75.50 

80.24 

4.74 

0.079 

8 

80.00 

74.72 

-5.28* 

0.045 

9 

74.75 

78.33 

3.58 

0.141 

10 

65.25 

71.50 

6.25 

0.058 

11 

77.50 

79.71 

2.21 

0.307 

12 

33.00 

33.57 

0.57 

0.463 

13 

82.69 

68.42 

-14.27 

0.063 

14 

81.67 

78.81 

-2.86 

0.204 

15 

75.75 

72.86 

-2.89 

0.196 

16 

74.00 

77.14 

3.14 

0.120 

17 

68.53 

75.48 

6.95* 

0.047 

18 

73.75 

70.24 

-3.51 

0.247 

19 

58.75 

66.67 

7.92* 

0.045 

20 

78.24 

77.22 

-1.01 

0.425 

21 

61.92 

59.00 

-2.92 

0.320 

22 

74.29 

80.48 

6.19 

0.056 

23 

77.00 

77.62 

0.62 

0.416 

24 

58.53 

64.76 

6.23 

0.104 

25 

76.84 

78.33 

1.49 

0.319 

26 

76.19 

79.29 

3.10 

0.135 

27 

78.57 

86.94 

8.37* 

0.008 

Class 

Avg. 

71.30 

72.78 

1.47 

0.160 

* Note: the increase is not significant at the 0.05 level. 


Student survey results showed increases in that the mate¬ 
rials were presented in logical order, greater material avail¬ 
ability and good use of class time. This is consistent with 
prior observations in the flipped classroom. However, stu¬ 
dent scores decreased in several areas. 
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Table 2 

Summary of Class Results 

Overall Class Mean Score Increase 

1.48 

Percentage 
Points * 

Percentage of Students With Significant 
Mean Score Improvements 

14.8% 

Percentage of Students with No 
Significant Mean Score Improvements 

81.5% 

Percentage of Students With Significant 
Mean Score Declines 

3.7% 

* indicates significant change in mean at the 0.05 level 
of significance 


Table 3 

Student Survey Results 

Question 

Results for 
Flipped 
Classroom 

The course material was presented in a 
logical order. 

10.0% 

increase 

The assigned readings helped me to learn 
the course material. 

10 . 0 % 

decrease 

The way the course was organized helped 
me to learn the material. 

11.0% 

decrease 

The instructor presented a variety 
of activities that got me involved in 
learning. 

9.0% 

decrease 

My scored on the exams, homework, 
and other graded assignments accurately 
reflected how much I actually learned. 

14.0% 

decrease 

The exams, homework, and other graded 
assignments were fair. 

4.0% 

decrease 

Materials for out-of-class learning 
activities were available when needed. 

3.3% 

increase 

The instructor consistently made good 
use of the entire class period 

10.5% 

increase 


Students’ scores decreased by 10% on the usefulness of 
the reading material. This may be due to the students 
thinking this referred only to the textbook and not the 
available online material. Therefore, on the next survey a 
more specific question needs to be crafted to evaluate the 
usefulness of the online material since 45.5% of the stu¬ 
dents specifically mentioned that the online material was 
one of the key components in their learning. 


It is interesting that there was an 11% decline on student 
perception that the format of the flipped classroom did 
not help them learn the material when 88% of the stu¬ 
dents specifically mentioned that the flipped classroom 
was effective in increasing their understanding of the ma¬ 
terial. There seems to be a numeric disconnect between 
the students written comments and their numeric score 
they selected. 

The key finding was that there was a 14% decrease in the 
student score that the testing used accurately measured 
what they learned and a 4% decline in test fairness scores. 
Although 70% of the final grade was measured using 
traditional multiple-choice exams, 30% of the grade was 
based on in-class presentations and project work. 

CONCLUSION 

Flipping the classroom did improve the test scores for 14% 
of the students and 88% of the students self-reported that 
the flipped classroom resulted in more effective learn¬ 
ing. However, overall class testing scores do not support 
that flipping the classroom improved the entire class. In 
fact, 81.5% of the students showed no improvement and 
3.7% showed lower test scores using the flipped classroom 
method. 

There appears to be conflicting data between self-reported 
learning and actual measured learning taking place. One 
reason may be the standardized testing method that is 
commonly employed. Does standardized multiple choice 
testing accurately measure deeper learning? Although this 
is a question relevant to the study, it is beyond the scope 
of the research. 

In general, the flipped classroom is not showing the sig¬ 
nificant increases in learning reported by the Byron High 
School study. Although flipping the classroom does work 
for some students, it does not appear to increase learning 
in all students as reported. 

Although this was a small study and sample size is a con¬ 
cern, the testing method was tightly controlled to measure 
only the increases in learning from the flipped classroom 
and not the pre-work or additional out of class work re¬ 
quired of the students. It makes logical sense, that any 
method that increases time spent learning would increase 
material understanding. It appears that the flipped class¬ 
room works by increasing the total time a student spends 
on the material. Therefore, the flipping the classroom 
method does not significantly increase learning but using 
technology to increase learning time does. 
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